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PREFACE 


TOP  200  USERS  SURVEY  REPORTING 
CONDUCTED  BY  THE  DEFENSE  TECHNICAL  INFORMATION  CENTER 

The  Defense  Technical  Information  Center  (DTIC)  has  surveyed  its 
registered  customers  since  1999.  There  are  two  surveys 
conducted : 

Top  200  Users.  Defined  in  terms  of  the  dollar  value  of  billing, 
number  of  documents  ordered,  and  downloads  from  Secure  STINET. 

All  200  users  are  surveyed.  Surveys  were  conducted  in  FY  2000, 

FY  2001  and  FY  2002 . 

Customer  Satisfaction  Survey.  A  random  sample  of  all  users  not 
included  in  the  Top  200  Survey.  Surveys  were  conducted  in  FY 
1999,  FY  2001  and  FY  2002. 

In  FY  2001,  selected  data  elements  from  both  surveys  were 
combined  into  the  2001  Annual  Survey  Composite  Data  Report.  A 
similar  combined  report  is  planned  for  FY  2002. 

The  DTIC  Performance  Report  to  the  Defense  Information  Systems 
Agency  (DISA)  contains  a  report  on  overall  customer  satisfaction. 
This  is  derived  from  the  Annual  Survey  Composite  Data  Report  and 
reflects  data  from  both  the  Top  200  Users  Survey  and  the  Customer 
Satisfaction  Survey. 


Top  200  Users  Survey  Results:  How  Do  We  Measure  Up? 


DTIC's  Top  200  customer  satisfaction  scores  continue  to 
rise  and  surpass  the  Federal  Government  American  Customer 
Satisfaction  Index  (ACSI)  baseline/benchmark  score.  See 
chart  below  for  illustration : 


Customer  Service 
Elements 

FY  2000 

Top  200  Users 
Survey 

FY  2001 

Top  200  Users 
Survey 

FY  2002 

Top  200  Users 
Survey 

DTIC  Composite  Scores 

76% 

77% 

82% 

ACSI  Federal  Government 
Benchmark* 

68.6% 

68.6% 

71.1% 

DTIC  Excels  by 

+7.4 

+8.4 

+10.9 

*ACSI  is  the  official  service  quality  benchmark  for  the  Federal  Government 

Note:  Data  relevant  to  this  chart  is  detailed  in  Part  II 
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FY  2002  Top  200  Users  Data  Summary 


Part  I :  Introduction 

Purpose:  In  June  2002,  the  Defense  Technical  Information  Center 

(DTIC)  conducted  the  FY  2002  Top  200  Users  Survey.  The  purpose 
of  the  survey  was  to  help  DTIC  (1)  gauge  the  level  of 
satisfaction  among  its  core  users  and  (2)  identify  possible  areas 
for  improving  our  products  and  services.  This  report  describes 
the  survey  process,  presents  its  results,  and  recommends  actions 
identified  by  survey  respondents.  Specifically,  this  report 
summarizes  the  results  of  the  FY  2002  Top  200  Users  Survey.  It 
also  compares  findings  from  the  FY  2000  and  FY  2001  Top  200  user 
data.  Collectively,  the  report  allows  us  to  evaluate  the 
following  specific  knowledge  objectives: 

■  To  improve  customer  retention 

■  To  determine  the  perceived  quality  of  products,  service  and 
customer  care 

■  To  indicate  trends  in  products,  services  and  customer  care 

■  To  benchmark  our  customer  satisfaction  results  with  other 
federal  government  agencies 

To  ensure  that  we  gained  fundamental  knowledge  objectives  from 
our  users,  the  FY  2002  Top  200  Users  Survey  was  specifically 
designed  by  querying  our  core  users  in  six  (6)  categories  listed 
below : 

■  Customer  Service  Experiences 

■  Global  Customer  Service  Performance 

■  DTIC  Online  Services 

■  DTIC  Products  and  Services 

■  User  Demographics 

■  Communication/Access  and  Information  Requirements 

With  regard  to  these  six  (6)  categories,  users  were  asked  to 
answer  questions  geared  toward  customer  service  factors: 
accessibility,  accuracy,  courtesy,  helpfulness,  knowledge, 
professionalism,  responsiveness,  and  speed  of  service.  Other 
questions  targeted  aspects  of  DTIC's  products  (i.e.,  customer 
involvement,  quality  and  access)  as  well  as  adequate,  useful  and 
prompt  information  distribution. 


Methodology:  Web-based  and  email  surveys  were  the  primary 

collection  methods  selected  for  this  effort.  One-on-one 
telephone  interviews  were  used  to  gather  contact  information  and, 
on  occasion,  conduct  telephone  surveys.  These  multiple 
collection  methods  were  selected  not  only  to  offer  our  users  a 
variety  of  survey  response  options,  but  also  to  increase  response 
rates . 

The  survey  population  universe  for  this  effort  was  218.  The  "Top 
200"  users  were  extracted  from  the  total  number  of  DTIC 
registered  users  based  on  total  dollar  amount  spent  on  DTIC 
products  and  services,  total  number  of  documents  ordered,  and 
total  number  of  document  downloads  (excluding  those  documents 
downloaded  from  Public  STINET) .  Those  users  with  the  highest 
figures  were  selected  and  considered  to  be  our  "core  users." 
Statistically,  the  survey  results  are  valid  with  a  survey 
population  of  218  users,  a  return  rate  of  15  percent,  a 
confidence  level  of  95  percent  and  a  35  percent  error  tolerance. 
After  4  contact  attempts,  only  144  users  were  reached.  The 
following  are  possible  reasons: 

•  Email  address  on  file  not  valid 

•  User  no  longer  at  email  address 

•  Increased  security  due  to  911  shut  down  of  agency  email 
systems 

•  Server  problems  for  recipient  or  originator 

Important  Note:  When  considering  the  results  of  Part  I  of  the 
survey  process,  it  is  important  to  recognize  not  only  the 
characteristics  of  the  core  customers  who  could  not  be  surveyed 
due  to  lack  of  valid  contact  information,  but  also  the  85  percent 
of  Top  200  registered  users  who  either  did  not  get  the  survey  or 
who  did  not  respond.  Analysis  of  the  survey  results  will  always 
need  to  be  tempered  by  considerations  of  who  did  and  who  did  not 
respond,  and  to  what  extent  those  who  responded  have  the  same 
characteristics  or  views  as  those  who  did  not  respond. 


Comparative  Analysis  of  DTIC  Users  FY  2001  Operational 
Data : 

The  Top  200  users  represent  the  single  best  source  for  our 
continued  growth.  To  put  this  into  perspective,  look  at  our  Top 
200  users'  impact  on  DTIC' s  FY  2001  business  results.  Core  users 
account  for  80  percent  of  DTIC's  total  billing,  more  than  half  of 
documents  ordered  and  nearly  half  of  documents  downloaded.  These 
significant  findings  are  illustrated  in  Fig  1.1.  The  chart 
breaks  down  the  total  FY  2001  billing  and  shows  the  importance  of 
core  customers  to  our  overall  operational  performance.  Note: 

Last  year's  (FY  2000)  core  users  represented  86  percent  of  DTIC's 
total  billing. 
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Comparative  Analysis  of  DTIC  Users  Annual  Billing,  Ordered,  Download 


Documents 


DTIC  Operational  Data 

All  DTIC 
Customers 

Top  200 

(219  actual  users) 

Overall  % 
(Scale:  1-100) 

FY  2001  Billing 

$547,141 

$440,124 

80% 

Documents  Ordered 

436,304 

247,378 

57% 

Documents  Downloads* 

5271 

2351 

45% 

*Note:  Excluding  those  documents  downloaded  from  Public  STINET 


Fig  1.1 


Overall  Findings 

(Data  relevant  to  the  overall  findings  below  are  detailed 

in  Appendix  B) 


Customer  Service  Issues 

♦  Eighty-two  percent  of  surveyed  respondents  reported  that  they 
were  satisfied  with  the  level  of  DTIC's  customer  care. 

( reference  page  41) 

♦  The  customer  satisfaction  service  score  has  shown  an  upward 
trend  in  comparison  to  previous  years,  (reference  page  41) 


Online  Services  (Public  STINET,  Secure  STINET,  Web-Enabled 
DROLS,  Classified  DROLS) 

♦  WED  and  Public  STINET  remain  the  most  widely  used  DTIC  Online 
Services,  (reference  page  42) 

♦  Overall  customer  satisfaction  score  for  DTIC  online  services 
remained  the  same  as  last  year  at  72  percent,  (reference  page 
47-48) 

♦  Secure  STINET  rated  the  highest  overall  in  customer  quality 
ranking  factors  (78.8  percent) .  (reference  page  48) 


Other  DTIC  Products  and  Services 

♦  Other  than  Online  Services,  the  most  widely  used  DTIC  product 
or  service  is  TR  Database  on  CD  ROM  (70  percent  usage  rate) . 
(reference  page  49) 


3 


♦  Overall  customer  satisfaction  score  for  "Other  DTIC  products 
and  services"  is  up  slightly  from  last  year  to  78  percent. 
(reference  page  50) 

♦  CAB  scored  the  highest  for  overall  performance  rating  for 
"Other  DTIC  products  and  services"  at  91  percent.  ( reference 
page  50) 


User  Demographics 

♦  The  majority  of  respondents  were  DOD  employees,  (reference  page 
52) 

♦  Seventy-one  percent  of  users  responding  were  Librarians. 
(reference  page  52) 

♦  The  vast  majority  of  core  users  reported  that  they  primarily 
act  as  information  providers  to  others,  (reference  page  53) 

♦  A  significant  majority  of  users  reported  being  registered  with 
DTIC  for  six  or  more  years,  (reference  page  53) 


Communication/Access 

♦  The  vast  majority  of  users  find  that  their  informational  needs 
are  being  met  by  searching  DTIC's  collection,  (reference  page 
53) 

♦  The  telephone  is  the  preferred  method  for  contacting  DTIC. 
(reference  page  53) 

♦  The  vast  majority  of  customers  reported  that  it  is  very 
important  to  extremely  important  to  speak  with  a  "live  person" 
when  calling  DTIC.  (reference  page  53) 

♦  More  than  half  of  the  Top  users  prefer  email  as  the  primary 
means  of  receiving  informational  updates,  (reference  page  55) 

♦  About  half  of  the  Top  users  acquire  DTIC  documents  by 
Internet/online  .  (reference  page  55) 
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Marketing  Communication 

♦  Users  reported  that  they  are  satisfied  that  DTIC  does  an 
adequate  job  in  communicating  (transmitting/mailing) 
information  about  new  changes  in  and/or  enhanced  products  and 
services  offerings,  training  opportunities,  etc.  (reference  page 
54-55) 

♦  The  majority  of  respondents  reported  reading  the  DTIC  DIGEST. 
(reference  page  55) 

♦  More  than  half  of  all  respondents  prefer  the  hardcopy  version 
of  the  DTIC  DIGEST  to  the  electronic  version,  (reference  page  55) 


General  Findings 

♦  Core  users  continue  to  highly  rate  the  importance  of  DTIC  to 
the  accomplishment  of  their  business  objectives,  (reference  page 
41 ) 

♦  Users  reported  that  DTIC  has  a  high  to  very  high  impact  in 
supporting  their  overall  mission,  (reference  page  41) 

♦  An  impressive  100  percent  of  core  user  respondents  would 
recommend  DTIC  to  colleagues,  (reference  page  41) 
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Part  II:  Global  Customer  Service 
Performance  Rating  Review 


Overall  Customer  Satisfaction  Performance  Ratings: 

DTIC's  overall  customer  care  performance  received  a  very  high 
mark.  A  majority  of  core  users  (82  percent)  reported  that  they 
were  satisfied  with  the  level  of  DTIC's  customer  care.  It  is 
also  important  to  note  that  the  FY  2002  aggregate  composite  score 
is  up  by  5  points,  from  77  percent,  since  last  year.  Fig  2.1 
below  reflects  customer  service  performance  over  a  three-year 
period: 


Comparative  Analysis  of  Customer 
Satisfaction  (Customer  Care)  Performance  Rating 


6 


DTIC’s  Overall  Customer  Care 
Quality  Factors/Elements 

FY  2001 

Top  200 

Users  Survey 

FY  2002 

Top  200 
Users  Survey 

Performance 

Gap 

Accessibility  (Easy  to  access  staff) 

63% 

72% 

+9 

Accuracy  (Provided  correct  info) 

80% 

82% 

+2 

Courtesy 

86% 

88% 

+2 

Helpfulness 

85% 

86% 

+1 

Knowledge 

81% 

83% 

+2 

Professionalism 

84% 

86% 

+2 

Responsiveness(Timely  response) 

69% 

78% 

+9 

Speed  of  Service 

70% 

78% 

+8 

Overall  Rating 

77% 

82% 

Fig  2.  2 


Comparative  Benchmarking-Comparison  to  American  Customer 

Satisfaction  Index  (ACSI) 

DTIC's  Customer  Service  Results: 

■  DTIC' s  customer  service  is  compared  to  the  2001  Federal 
Government  American  Customer  Satisfaction  Index  (ACSI) ,  which 
surveys  53  different  customer  groups  from  39  different  Federal 
government  agencies . 

■  The  2001  December  aggregate  ACSI  score  for  the  Federal 
Government  is  71.1  percent,  up  3.5  points  from  68.6  percent 
since  last  year. 

■  DTIC's  composite  data  was  matched  for  comparability  with  the 
common  ASCI  index  scoring  factors. 

■  DTIC' s  FY  2002  Overall  Top  200  Users  Customer  Satisfaction 
composite  score  is  82  percent. 

■  When  measured  in  terms  of  the  2001  ACSI,  it  is  above  the 
established  federal  baseline/benchmark  score  by  +10.9  points. 

Note:  FY  2002  DTIC  baseline  customer  satisfaction  performance 
score  is  82  percent  (FY  2002  Top  200  users  and  FY  2002 
Customer  Satisfaction  integrated  composite  score) . 


Customer  Service 
Elements 

FY  2001 

Top  200  Users 
Survey 

FY  2002 

Top  200  Users 
Survey 

DTIC  Composite  Scores 

76% 

77% 

82% 

ACSI  Federal  Government 
Benchmark* 

68.6% 

68.6% 

71.1% 

DTIC  Excels  by 

+7.4 

+8.4 

+10.9 

*ACSI  is  the  official  service  quality  benchmark  for  the  Federal  Government 

Fig  2.3 
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Importance  of  DTIC  to  Accomplishment  of  Business  Objectives 
and  Mission  Support: 


Core  users  continue  to  highly  rate  the  importance  of  DTIC  to  the 
accomplishment  of  their  business  objectives  (90  percent) .  This 
finding  can  be  supported  by  the  fact  that  85  percent  of  core 
users  communicated  the  importance  of  DTIC's  impact  in  supporting 
their  overall  mission.  See  graphs  below  (Fig  2.4  and  2.5)  for 
more  details. 
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FY  2002  Importance  of  DTIC  to 
Accomplishment  of  Business 
Objectives 


0- 

FY  2000  Top  200 

FY  2001  Top  200 

FY  2002  Top  200 

User  Survey  % 

User  Survey  % 

User  Survey  % 

■  Very  to  Extremely  Important 

92 

86 

90 

□  Important 

7 

13 

8 

■  Not  Very  Important  and/or 

1 

1 

2 

Not  At  All  Important 

Fig  2.4 
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FY  2002  Impact  of  DTIC 
Supporting  Users  Overall  Mission 


u- 

FY  2001  Top  200  User 
Survey  % 

FY  2002  Top  200  User 
Survey  % 

■  High  to  Very  High  Impact 

79 

85 

□  Medium  Impact 

17 

10 

■  Low  to  Very  Low  Impact 

4 

5 

Fig  2.5 


Recommending  DTIC  Product (s)  and/or  Service (s)  to 
Colleagues : 

An  impressive  100  percent  of  core  user  respondents  would 
recommend  DTIC  to  colleagues.  This  high  number  is  consistent 
with  the  supported  written  comments  from  respondents  indicating 
their  need  and  support  for  DTIC's  type  of  services.  See  Part  VII 
for  a  sampling  of  supportive  user  comments. 


Comparative  Analysis  on 
Recommendations  of  DTIC  to 
_ Colleagues _ 


1 00  -| 

§>  80- 
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<D 

“  60- 
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0- 

FY  2001  Top  200  User  Survey  % 

FY  2002  Top  200  User  Survey  % 

■  Yes 

98 

100 

□  No 

2 

0 

■  Not  Sure 

0 

0 

Note:  FY  2000  -  no  comparable  data  are  available 

Fig  2.6 
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Delivery  Processes: 


Users  responding  to  the  FY  2002  Top  200  Users  Survey  reported  the 
following  regarding  satisfaction  with  DTIC  meeting  anticipated 
delivery  schedules: 


0 

o> 

re 

c 

re 

o 

0 

CL 


90 

80 

70 

60 

50 

40 

30 

20 

10 

0 


DTIC  Comparative  Analysis 


Customer  Satisfaction  with 
Meeting  Delivery  Schedules 


- 88 - 88 - 

FY  2001  Top  200  Users  Survey 

% 


88 


FY  2002  Top  200  Users 
Survey  % 


88 


Note:  FY  2000  -  no  comparable  data  are  available 

Fig  2.7 


Customer  Satisfaction  with  Meeting 
Delivery  Schedules 

FY  2001 

Top  200  Users 
Survey 

FY  2002 

Top  200  Users 
Survey 

Every  time  to  most  of  the  time 

88% 

88% 

About  Half  the  time 

12% 

12% 

Never  to  less  than  half 

0% 

0% 

Download  documents  themselves  (Do  not 
order  documents) 

N/A* 

0% 

‘Data  not  gathered  in  previous  survey 

Fig  2.8 


There  was  some  concern  that  postal  operations  disrupted  by  the 
anthrax  contamination  last  fall  might  have  reduced  satisfaction 
with  DTIC  delivery  of  reports,  but  this  did  not  seem  to  have  an 
impact . 


It  is  interesting  that  no  Top  200  respondents  reported 
downloading  documents  instead  of  ordering  hardcopy,  though  they 
did  download  documents  in  addition  to  ordering  hardcopy. 


10 


Part  III:  DTIC  Online  Services 


Based  on  results  of  the  FY  2002  Top  200  Users  Survey,  the  most 
widely  used  online  services  DTIC  has  to  offer  are  WED  and  Public 
STINET  as  shown  by  an  84  percent  usage  profile  for  both  services. 
Results  from  the  FY  2001  Top  200  Users  Survey  indicated  product 
usage  profile  remained  basically  the  same  with  WED  having  the 
greatest  number  of  users  (76  percent)  followed  by  Public  STINET 
(68  percent) .  Of  the  users  responding,  usage  of  all  online 
services  has  increased.  Most  services  are  showing  at  least  an 
8  percentage  point  increase  in  usage  with  Secure  STINET  showing 
the  greatest  increase  (16.5  percentage  point  increase) . 


Online  Service  Usage  Profile 


Service 

FY  2001 

Top  200  Users 
Survey  Usage 

FY  2002 

Top  200  Users 
Survey  Usage 

%  Point 

Increase 

Public  STINET 

67.7% 

83.7% 

+16.0 

Secure  STINET 

63.0% 

79.5% 

+16.5 

WED  (formerly 
Unclassified  DROLS) 

76.3% 

84.4% 

+8.1 

Classified  DROLS 

23.7% 

37.9% 

+14.2 

Note:  Comparable  data  are  not  available  for  FY  2000 


Fig  3.1 


For  those  users  responding  to  the  survey,  detailed  products  and 
services  usage  is  provided  as  follows  including  a  breakdown  for 
the  top  3  user  groups. 

Note:  The  FY  2002  top  job  positions  in  which  the  Top  200  User 

survey  respondents  categorized  themselves  are  (1)  Librarian,  (2) 
Technical  Information  Specialist,  (3)  Researcher/Analyst/Program 
Manager,  and  (4)  Scientist.  Both  Researcher/Analysts  and 
Scientists  have  the  same  score  (4  percent) .  Since  comparisons  of 
the  top  3  job  positions  have  been  reported  in  recent  years, 
survey  results  are  being  reported  for  Librarians,  Technical 
Information  Specialists,  and  Researcher/Analysts  in  order  to  have 
comparable  data.  See  Part  V  -  Demographics  for  details. 
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Users  responding  to  the  FY  2002  Top  200  Users  Survey  were  asked 
to  rate  the  level  of  online  service  using  several  quality 
factors.  The  following  identifies  the  online  service  quality 
rating  segmented  by  the  top  three  user  groups.  Librarians, 
Technical  Information  Specialists,  and  Researcher/Analysts. 
Note:  For  Figures  3.4  through  3.9,  and  4.2  through  4.10,  the 

count  of  respondents  in  some  categories  is  small,  at  times 
resulting  in  large  percentage  shifts. 

Accessibility  Overall  Ratings : 

•  76  percent  of  Users  rated  accessibility  as  "Very  Good"  to 
"Excellent" 

•  15  percent  as  "Good" 

•  9  percent  as  "Fair"  to  "Poor" 


Analysis  of  DTIC's  Online  Services  Performance  Rating  - 
Accessibility  (Access  to  site) 


FY  2002  Top  200  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

76% 

15% 

9% 

Librarian 

74% 

19% 

7% 

Technical  Information  Specialist 

83% 

0% 

17% 

Researcher/Analyst 

63% 

37% 

0% 

Fig  3.4 


Ease  of  Use  (Navigating  the  site)  Overall  Ratings: 

•  65  percent  of  users  rated  navigation  as  "Very  Good"  to 
"Excellent" 

•  23  percent  as  "Good" 

•  12  percent  as  "Fair" 


Analysis  of  DTIC's  Online  Services  Performance  Rating  -  Ease  of 

Use  (Navigating  the  Site) 


FY  2002  Top  200  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

65% 

23% 

12% 

Librarian 

58% 

31% 

11% 

Technical  Information  Specialist 

83% 

0% 

17% 

Researcher/Analyst 

88% 

12% 

0% 

Fig  3.5 
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Content  Overall  Rating: 

•  78  percent  of  users  rated  content  as  "Very  Good"  to 
"Excellent" 

•  13  percent  as  "Good" 

•  9  percent  as  "Fair" 


Analysis  of  DTIC's  Online  Services  Performance  Rating  -  Content 


Fig  3.6 

Organization  Overall  Rating: 

•  63  percent  of  users  rated  organization  as  "Very  Good"  to 
"Excellent" 

•  26  percent  as  "Good" 

•  11  percent  as  "Fair" 


Analysis  of  DTIC's  Online  Services  Performance  Rating  - 

Organization 


FY  2002  Top  200  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

63% 

26% 

11% 

Librarian 

56% 

35% 

9% 

Technical  Information  Specialist 

78% 

5% 

17% 

Researcher/Analyst 

1 00% 

0% 

0% 

Fig  3.7 


Usability  Overall  Rating: 

•  77  percent  of  users  rated  usability  as  "Very  Good"  to 
"Excellent" 

•  13  percent  as  "Good" 

•  10  percent  as  "Fair" 


Analysis  of  DTIC' s  Online  Services  Performance  Rating  -  Usability 


FY  2002  Top  200  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

77% 

13% 

10% 

Librarian 

74% 

18% 

7% 

Technical  Information  Specialist 

83% 

0% 

17% 

Researcher/Analyst 

1 00% 

0% 

0% 
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Fig  3.8 


Customer  Support  Overall  Rating: 

•  69  percent  of  users  rated  customer  support  as  "Very  Good"  to 
"Excellent" 

•  8  percent  as  "Good" 

•  23  percent  as  "Fair" 


Analysis  of  DTIC' s  Online  Services  Performance  Rating  -  Customer 

Support 


FY  2002  Top  200  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

69% 

8% 

23% 

Librarian 

66% 

10% 

23% 

Technical  Information  Specialist 

37% 

10% 

53% 

Researcher/Analyst 

63% 

0% 

37% 

Fig  3.9 


Comparative  Analysis  of  DTIC's  Individual  Online  Services  Rating 


FY  2002  Top  200  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Public  STINET 

71% 

20% 

9% 

Secure  STINET 

83% 

12% 

5% 

WED 

75% 

15% 

10% 

Classified  DROLS 

56% 

19  % 

25% 

Overall  Rating 

1  72% 

16% 

12% 

FY  2001  Top  200  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Public  STINET 

69% 

24% 

7% 

Secure  STINET 

71% 

23% 

6% 

WED 

79% 

16% 

5% 

Classified  DROLS 

68% 

18% 

14% 

Overall  Rating 

72% 

20% 

8% 

Fig  3.10 
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Performance  of  Individual  Online  Services : 


Figures  3.11  to  3.14  depict  the  overall  rating  of  various  quality 
factors  for  each  online  service  for  the  FY  2002  Top  200  Users 
Survey . 

Public  STINET  Quality  Performance  Rating 


0  10  20  30  40  50  60  70  80  90  100 

(%) 


Fig  3.11 


Secure  STINET  Quality  Performance  Rating 

Accessibility 
Ease  of  Use 
Content 
Organization 
Usability 
Customer  Support 
Overall  Average 

0  10  20  30  40  50  60  70  80  90  100 

(%) 

Fig  3.12 
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WED  Quality  Performance  Rating 


0  10  20  30  40  50  60  70  80  90  100 

(%) 


Fig  3.13 


Classified  DROLS  Quality  Performance  Rating 

Accessibility 
Ease  of  Use 
Content 
Organization 
Usability 
Customer  Support 
Overall  Average 

0  10  20  30  40  50  60  70  80  90  100 

(%) 

Fig  3.14 
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Part  IV:  DTIC ' s  Other  Products  and  Services  Usage  Profile 


The  most  widely  used  product  or  service  DTIC  has  to  offer  is  the 
TR  Database  on  CD-ROM  as  shown  by  a  70  percent  usage  profile, 
followed  by  the  DTIC  Review  and  ECAB  at  58  percent  and  50  percent 
respectively.  Although  the  data  for  the  FY  2001  Top  200  Users 
Survey  is  limited,  the  results  indicate  product  usage  profiles 
remained  basically  the  same  with  TR  Database  on  CD  ROM  and  ECAB 
having  the  greatest  number  of  users  as  can  be  seen  in  Figure  4.1. 
Of  the  users  responding,  it  appears  usage  of  all  DTIC  products 
and  services  has  increased.  Most  products  are  showing  at  least  a 
5  percentage  point  increase  in  product/service  usage  with  the 
DTIC  Review  showing  the  greatest  increase  (34.1  percentage  point 
increase) . 


Products  and  Services  Usage  Profile 


Service/Usage 

FY  2001 

Top  200  Users 
Survey 

FY  2002 

Top  200  Users 
Survey 

%  Point  Increase 

ADD 

28.9% 

34.5% 

+5.6 

CAB 

30.1% 

40% 

+9.9 

ECAB 

40.5% 

50% 

+9.5 

Research  Summaries 

N/A* 

43.3% 

N/A* 

Automatic  Magnetic  Tape 
Dist. 

N/A* 

0% 

N/A* 

TR  Database  CD-ROM 

45.1% 

70.3% 

+25.2 

Retrospective 

Bibliography  Magnetic 

Tape 

N/A* 

3.8% 

N/A* 

The  DTIC  Review 

24.4% 

58.1% 

+33.7 

TRAIL 

N/A* 

28% 

N/A* 

‘Data  not  gathered  in  previous  survey 


Fig  4.1 

For  those  users  responding  to  the  survey,  detailed  product  and 
service  usage  data  is  provided  as  follows  including  a  breakdown 
for  the  top  3  job  positions. 

Note:  The  FY  2002  top  job  positions  in  which  the  Top  200  Users 

survey  respondents  categorized  themselves  are  (1)  Librarian,  (2) 
Technical  Information  Specialist,  (3)  Researcher/Analyst/Program 
Manager,  and  (4)  Scientist.  Both  Researcher/Analysts  and 
Scientists  have  the  same  score  (4  percent) .  Since  comparisons  of 
the  top  3  job  positions  have  been  reported  in  recent  years, 
survey  results  are  being  reported  for  Librarians,  Technical 
Information  Specialists,  and  Researcher/Analysts  in  order  to  have 
comparable  data.  See  Part  V  -  Demographics  for  details. 
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Note:  %  Totals  may  not  equal  100% 

Fig  4.2 


Overall  Product/Service  Quality  and  Performance: 

For  the  FY  2002  users  responding  to  the  survey,  the  composite 
products  and  services  overall  quality  satisfaction  profile  showed 
75  percent  rating  "Very  Good"  to  "Excellent, "  20  percent  as 
"Good, "  and  5  percent  as  "Fair"  to  "Poor, "  for  an  overall  rating 
of  78  percent.  Ratings  were  similar  in  FY  2001  showing  76 
percent  as  "Very  Good"  to  "Excellent,"  18  percent  as  good,  and  6 
percent  as  "Fair"  to  "Poor."  The  overall  quality  satisfaction 
rating  for  the  FY  2001  survey  was  75  percent.  It  should  be  noted 
that  the  composite  rating  for  FY  2001  included  only  four  DTIC 
products  and  services:  ADD,  CAB,  ECAB/ECAB  Docs  and  TR  Database 
on  CD  ROM.  The  products  and  services  list  comprising  the 
composite  rating  for  FY  2002  was  more  extensive  (seven  products  & 
services  rated) . 
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Comparative  Analysis  of  Overall 
Product/Service  Quality 


u  - 

FY  2001  Top  200  Users 
Survey  % 

FY  2002  Top  200  Users 
Survey  % 

■  Very  Good  to  Excellent 

76 

75 

□  Good 

18 

20 

■  Fair  to  Poor 

6 

5 

Note:  Comparable  data  are  not  available  for  FY  2000 

Fig  4.3 


User  Satisfaction  with  Individual  Products  and  Services: 


Individually,  in  FY  2002  the  products  and  services  receiving  the 
highest  marks  for  overall  quality  satisfaction  were  CAB  (91 
percent),  ADD  (89  percent)  and  the  DTIC  Review  (80  percent) . 
Individual  products  and  services  ratings  were  not  available  for 
FY  2001  and  1999.  Detailed  information  for  several  DTIC  products 
and  services  is  provided  below.  Note  that  although  the  ratings 
by  Researcher/Analysts  are  high  for  most  DTIC  Products  and 
Services,  the  data  typically  reflects  the  opinion  of  only  one 
respondent . 

Automatic  Document  Distribution  (ADD) : 

•  82  percent  of  users  rated  ADD  as  "Very  Good"  to  "Excellent" 

•  18  percent  as  "Good" 

•  0  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  ADD 


FY  2002  Top  200  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

81.8% 

18.2% 

0% 

Librarian 

75.0% 

25.0% 

0% 

Technical  Information  Specialist 

1 00% 

0% 

0% 

Researcher/Analyst 

0% 

0% 

0% 

Fig  4.4 


20 


Current  Awareness  Bibliography  (CAB) : 

•  92  percent  of  users  rated  CAB  as  "Very  Good"  to  "Excellent" 

•  8  percent  as  "Good" 

•  0  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  CAB 


FY  2002  Top  200  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

92.3% 

7.7% 

0% 

Librarian 

90% 

10% 

0% 

Technical  Information  Specialist 

1 00% 

0% 

0% 

Researcher/Analyst 

0% 

0% 

0% 

Fig  4.5 


Electronic  Current  Awareness  Bibliography  (ECAB) : 

•  67  percent  of  users  rated  ECAB  as  "Very  Good"  to  "Excellent" 

•  13  percent  as  "Good" 

•  20  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  ECAB 


FY  2002  Top  200  Users  Survey 

Favorable 

Neutral 

Unfavorable 

Overall 

66.7% 

13.3% 

20.0% 

Librarian 

72.7% 

9.1% 

18.20% 

Technical  Information  Specialist 

66.7% 

0% 

33.3% 

Researcher/Analyst 

0% 

1 00% 

0% 

Fig  4.6 


Research  Summaries  (RS) :  NOTE:  Some  users  of  this  Current 
Awareness  product  may  have  confused  it  with  RS  data  in  other 
databases.  There  were  14  respondents. 

•  57  percent  of  users  rated  RS  as  "Very  Good"  to  "Excellent" 

•  43  percent  as  "Good" 

•  0  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  - 

Summaries 

Research 

1  FY  2002  Top  200  Users  Survey 

Favorable% 

Neutral% 

Unfavorable% 

r 

||  Overall 

1  57.1% 

|  42.9% 

1  o%  | 

Librarian 

37.5% 

62.5% 

0% 
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Technical  Report  (TR)  Database  CD-ROM: 

•  81  percent  of  users  rated  the  TR  Database  on  CD-ROM  as  "Very 
Good"  to  "Excellent" 

•  19  percent  as  "Good" 

•  0  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  TR  Database 

CD-ROM 


FY  2002  Top  200  Users  Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

80.8% 

19.2% 

0% 

Librarian 

76.5% 

23.5% 

0% 

Technical  Information  Specialist 

80% 

20% 

0% 

Researcher/Analyst 

1 00% 

0% 

0% 

Fig  4.8 


The  DTIC  Review: 

•  88  percent  of  users  rated  the  DTIC  Review  as  "Very  Good"  to 
"Excellent" 

•  12  percent  as  "Good" 

•  0  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  The  DTIC 

Review 


FY  2002  Top  200  Users  Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

88.2% 

11.8% 

0% 

Librarian 

83.3% 

16.7% 

0% 

Technical  Information  Specialist 

1 00% 

0% 

0% 

Researcher/Analyst 

1 00% 

0% 

0% 

Fig  4.9 


Technical  Report  Automated  Information  List  (TRAIL) : 

•  57  percent  of  users  rated  TRAIL  as  "Very  Good"  to  "Excellent" 

•  29  percent  as  "Good" 

•  14  percent  as  "Fair" 


Analysis  of  Products  &  Services  Performance  Rating  -  TRAIL 


FY  2002  Top  200  Users  Survey 

Favorable% 

Neutral% 

Unfavorable% 

Overall 

57.1% 

28.6% 

14.3% 

Librarian 

40% 

40% 

200% 

Technical  Information  Specialist 

1 00% 

0% 

0% 

Researcher/Analyst 

1 00% 

0% 

0% 

Fig  4.10 
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Part  V:  User  Demographics 


Job  Position  Status  and  Organization  Type : 

The  customer  profile  in  this  survey  represents  five  (5) 
organizational  groups  and  seven  (7)  job  positions,  the 
distribution  of  which  is  shown  in  Figures  5.1  and  5.2.  Aspects 
of  the  FY  2001  and  FY  2002  Top  200  Users  Surveys  showed  similar 
organizational  status  percentages,  indicating  that  the  majority 
of  respondents  were  affiliated  with  the  DOD  community  (60 
percent) .  The  response  rate  for  the  Industry  (Gov't  Contractors, 
Corporate,  etc.)  was  12  percentage  points  lower  than  last  year. 
Overall,  the  respondent's  top  four  (4)  organization  types  remain 
consistent  from  previous  surveyed  years. 


User  Organization  Type 


Organization 

FY  2000 

Top  200  Users 
Survey 

FY  2001 

Top  200  Users 
Survey 

FY  2002  Top 
200  Users 
Survey 

DOD-Military  Services 
Agencies 

51% 

57% 

60% 

Gov't.  Contract/Industry 

45% 

33% 

21% 

Non-DOD  Federal  Gov't. 

2% 

5% 

8% 

Academia:  Coliege/Univ. 

2% 

5% 

8% 

HBCU/MI 

N/A* 

N/A* 

0% 

Other 

0% 

0% 

3% 

*Data  not  gathered  in  previous  survey 


Fig  5.1 

FY  2002  findings  show  a  slight  change  with  last  year's  top  three 
(3)  job  position  categories,  noting  that  this  year  we  have  no 
Engineers  as  respondents.  This  year,  a  majority  of  the 
respondents  classified  their  job  position  as  Librarian  (71 
percent)  with  Technical  Information  Specialists  following  at  13 
percent.  FY  2002  survey  findings  show  Librarians  represent  a  12 
percentage  point  increase  in  comparison  to  FY  2001  survey 
findings.  Researcher/Analysts  and  Scientists  were  listed  as  the 
next  two  job  positions  with  4  percent  of  respondents  each. 
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Job  Position 

FY  2000 

Top  200  Users 
Survey 

FY  2001 

Top  200  Users 
Survey 

FY  2002 

Top  200  Users 
Survey 

Librarian 

72%* 

57%* 

71  %* 

Technical  Info  Specialist 

11%* 

1 2%* 

1 3%* 

Researcher/Analyst/Prog  ram 
Manager 

7%* 

10% 

4%* 

Scientist 

1% 

3% 

4%* 

Engineer 

1% 

11%* 

0 

Contracting  Officer 

1% 

0 

0 

Resource 

Management/Financial/ 

Budget  Analyst 

1% 

0 

2% 

Security 

1% 

5% 

N/A“ 

Other 

5% 

2% 

6% 

‘Note:  Highlights  indicate  the  Top  200  users  job  position  of  each  surveyed  year 
“Data  not  gathered  in  previous  survey 


Fig  5.2 

Intermediary  vs .  End  Users : 

The  two  categories  listed  in  the  chart  below  reflect  areas  on  how 
our  users  acquire  scientific  and  technical  information.  The  FY 
2002  Top  200  Users  Survey  shows  a  continuing  trend  of  our  users 
primarily  acting  as  information  providers  to  others  as 
illustrated  in  Fig  5.3.  It  is  noteworthy  that  FY  2002  survey 
data  showed  94  percent  of  the  DOD  respondents  and  73  percent  of 
the  Industry  respondents  overwhelmingly  reported  themselves  as 
information  providers  to  others. 
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Time  as  a  Registered  User: 


There  has  been  a  dramatic  change  between  FY  2001  and  FY  2002  with 
respect  to  how  long  respondents  have  been  registered  with  DTIC. 
The  results  from  the  FY  2002  Top  200  Users  Survey  show  the 
majority  of  respondents  have  been  registered  users  for  "six  or 
more  years"  which  also  reflects  the  FY  2001  survey.  However,  a 
comparison  of  the  FY  2002-FY  2001  results  shows  an  interesting 
variation;  first  of  all,  findings  show  a  41  percent  increase  in 
users  responding  to  "6  or  more  years."  Secondly,  findings  show  a 
32  percent  decrease  in  users  responding  to  "less  than  1  year." 

It  is  unclear  why  there  is  a  significant  deviation  between  the 
two  surveyed  years.  It  would  be  reasonable,  however,  to  expect 
that  long  term  users  would  have  higher  response  rates  than  users 
who  have  been  registered  with  DTIC  for  a  shorter  period.  The 
data  displayed  in  Fig  5.4  below  defines  respondent's  time  as  a 
registered  DTIC  user. 


Part  VI:  Communications,  Access  and  Information 

Requirements 

Information  Needs : 

The  vast  majority  of  users  (96  percent)  reported  a  high  level  of 
satisfaction  with  their  information  needs  being  met  by  searching 
DTIC's  collections.  However,  they  do  express  continual  need  for 
full  text  online  publications.  It  is  also  interesting  to  note 
that  this  year,  respondents  appear  to  be  asking  for  older 
reports;  some  gave  specifics,  but  the  majority  responded  with 
just  a  request  for  older  information.  See  below  for  the  top  four 
informational  needs  expressed  by  the  respondents. 


■  Full  text  access 

■  Older  full  text  online  publications 

■  Old  Publications- (on  Biological/Chemical  Warfare,  Nuclear 
Weapons ) 

■  Homeland  Security 

Note:  These  four  (4)  subject  areas  (information  needs) 

identified  above  were  the  result  of  respondent's  most  frequently 
mentioned  write-ins . 

Communication  Preferences  and  Satisfaction: 


Aspects  of  the  FY  2001  and  FY  2002  Top  200  Users  survey  findings 
show  similar  medium  preference  percentages  from  the  respondents. 
In  fact,  the  survey  data  appears  to  be  very  similar  to  last 
year's  results.  The  majority  of  respondents  still  use  the 
telephone  as  the  preferred  method  (62  percent)  for  contacting 
DTIC.  It  is  important  to  note  that  despite  the  slight  decrease 
in  internet/online  preference,  it  still  remains  the  second 
preferred  communication  contact  method. 


Comparative  Analysis  of  Access 
Medium  (User  Contact  Preference) 
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0- 

FY  2000  Top 
200  Users 
Survey  % 

FY  2001  Top 
200  User 
Survey  % 

FY  2002  Top 
200  Users 
Survey  % 

■  Telephone  (Commerical/DSN) 

86 

60 

62 

□  Internet/Online 

6 

23 

17 

■  Email 

4 

10 

11 

□  Fax 

4 

6 

8 

■  Others-Mail 

N/A* 

1 

2 

uata  were  not  gathered  in  previous  survey 


Mg  b.i 
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Importance  of  Talking  to  a  "Live  Person" : 


FY  2002  survey  findings  continue  the  trend  of  respondent's 
preference  to  speak  to  a  "live  person"  when  contacting  DTIC 
(84  percent) .  This  finding  provides  additional  support  to  the 
concept  that  the  majority  of  our  respondent's  preferred  "live 
person"  telephone  contacts. 
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Comparative  Analysis  on  Importance  of 
Talking  to  a  Live  Person 


IS 


u- 

FY  2000  Top  200 
User  Survey  % 

FY  2001  Top  200 
User  Survey  % 

FY  2002  Top  200 
User  Survey  % 

□  Very  to  Extremely  Important 

89 

82 

84 

□  Important 

9 

11 

6 

□  Not  Very  Important  and/or 

Not  At  All  Important 

2 

7 

10 

Fig  6.2 


Effectiveness  of  Corporate  Communications : 

The  data  contained  in  Fig  6.3  below  describes  the  effectiveness 
of  DTIC's  marketing  communication  efforts.  In  other  words,  "How 
well  do  we  get  the  word  out?"  When  users  were  asked  this 
question,  a  majority  of  our  respondents  (88  percent)  revealed 
that  DTIC  does  an  adequate  job  in  communicating 
(transmitting/mailing)  information  about  new  changes  in  and/or 
enhanced  products  and  services  offerings,  training  opportunities, 
etc.  This  finding  is  consistent  from  previous  surveyed  years.  In 
the  past  two  years,  DTIC  has  sharply  reduced  its  use  of  paper- 
based  communication  through  the  mail. 
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Comparative  Analysis  on  Effectiveness  of  Corporate  Communication 


FY  2000  Top  200  Users  Survey 

Strongly  Agree 
or  Agree 

Strongly  Disagree  or 
Disagree 

No  Opinion 

New  Product  /Service  Changes 

95 

1 

4 

Product  /Service  Changes 

94 

3 

3 

Ordering  Documents 

81 

7 

12 

Training  Opportunities 

75 

3 

22 

Submitting  Documents 

48 

4 

48 

Overall  Rating 

78% 

4% 

18% 

FY  2001  Top  200  Users  Survey 

Strongly  Agree 
or  Agree 

Strongly  Disagree  or 
Disagree 

No  Opinion 

New  Product  /Service  Changes 

92 

2 

6 

Product/Service  Changes 

91 

6 

3 

Submitting  Documents 

63 

7 

30 

Training  Opportunities 

87 

0 

13 

Ordering  Documents 

93 

0 

7 

System  Disruptions/Downtime 

82 

2 

16 

Overall  Rating 

85% 

3% 

12% 

FY  2002  Top  200  Users  Survey 

Strongly  Agree 
or  Agree 

Strongly  Disagree  or 
Disagree 

No  Opinion 

New  Product  /Service  Changes 

94 

2 

4 

Product/Service  Changes 

90 

2 

8 

Submitting  Documents 

68 

27 

7 

Training  Opportunities 

90 

8 

2 

Ordering  Documents 

92 

6 

2 
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Fig  6.3 

When  asked  to  identify  how  they  would  most  like  to  be  informed 
about  DTIC  products/services,  updates,  etc.,  users  responded  as 
follows : 

■  58  percent  preferred  Email 

■  27  percent  preferred  Electronic  Mailing  List  (listservs) 

■  9  percent  preferred  Mail 

■  4  percent  preferred  Web  Site 

■  2  percent  preferred  Fax 


It  should  be  noted  that  "submitting  documents"  is  often  done  by  a 
different  organizational  element  than  that  represented  by  the 
respondent . 
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